fix: cap adversarial-review prompt at 800KB with UTF-8-safe fallback chain#314
Open
cardene777 wants to merge 1 commit into
Open
fix: cap adversarial-review prompt at 800KB with UTF-8-safe fallback chain#314cardene777 wants to merge 1 commit into
cardene777 wants to merge 1 commit into
Conversation
…chain Heavy-tier adversarial reviews crash with `Input exceeds the maximum length of 1048576 characters.` when `buildAdversarialReviewPrompt` interpolates a near-256KB diff alongside the standard template (closes openai#11 follow-up). This change adds a `MAX_PROMPT_BYTES = 800 * 1024` byte cap to `buildAdversarialReviewPrompt` (now exported) and a three-step fallback chain: 1. Initial render — return verbatim if within cap. 2. Lightweight fallback — switch guidance to "Inspect the target diff yourself" and replace `REVIEW_INPUT` with summary + changedFiles. 3. Hard truncation — UTF-8-safe byte truncation with explicit `[content truncated to fit prompt size limit]` marker. The truncation routine walks back from UTF-8 continuation bytes (0x80-0xBF) so multi-byte sequences are never split mid-character. `Buffer.byteLength` is used throughout to measure UTF-8 bytes, not UTF-16 code units. `isDirectExecution()` wraps `main()` so importing the module from tests no longer runs the CLI as a side effect. Comparison uses `fs.realpathSync.native()` on both `process.argv[1]` and `fileURLToPath(import.meta.url)` so symlinked install paths (plugin cache, macOS `/var` vs `/private/var`) still match. Lexical comparison remains as a fallback when realpath throws. Adds `tests/codex-companion.test.mjs` covering small / at-limit / 1MB lightweight fallback / 5MB truncation / 750KB multibyte input. Refs: cardene777/claude-config#1467
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Heavy-tier
adversarial-reviewinvocations crash withInput exceeds the maximum length of 1048576 characters.whenbuildAdversarialReviewPromptinterpolates a 256KB diff alongside the standard template. This is the follow-up scenario surfaced after PR #179 (256KB inline-diff cap) — that fix preventsENOBUFS, but the assembled prompt can still exceed the Codex thread input cap on its own.This PR adds a hard
MAX_PROMPT_BYTES = 800 * 1024byte cap on the assembled prompt (template + placeholders + content), plus a 3-step fallback chain that always stays under the cap.Changes
plugins/codex/scripts/codex-companion.mjsbuildAdversarialReviewPromptis nowexported and wraps its output in a 3-step fallback chain:Buffer.byteLength(prompt, "utf8") <= MAX_PROMPT_BYTES."The repository context below is a lightweight summary. Inspect the target diff yourself..."(mirrorsbuildAdversarialCollectionGuidance({ includeDiff: false })) and rebuildREVIEW_INPUTfromsummary/changedFiles/fileCount(no diff body).(buf[end] & 0xc0) === 0x80) to preserve valid sequences, and append\n\n[content truncated to fit prompt size limit]\nso the reviewer model can see truncation occurred.isDirectExecution()wrapsmain()so thatimporting the module from tests no longer runs the CLI as a side effect. Comparison usesfs.realpathSync.native()on bothprocess.argv[1]andfileURLToPath(import.meta.url)so symlinked install paths (plugin cache, macOS/varvs/private/var) still match the entry. Lexical comparison remains as a fallback when realpath throws.tests/codex-companion.test.mjs(new)Five
node:testcases covering:Buffer.byteLength === 800 * 1024, full content preservedxcontentxcontenttruncatedmarkerあcontentTest plan
node --check plugins/codex/scripts/codex-companion.mjsnode --test tests/codex-companion.test.mjs→ 5 / 5 passnode --test tests/*.test.mjs→ 82 / 91 pass (9 pre-existing failures onmain807e03a: 5 intests/runtime.test.mjssetup …, 3 intests/commands.test.mjsstatus …/result …, 1 intests/state.test.mjsresolveStateDir …— all reproduce on a clean upstream checkout and are unrelated to this PR)node plugins/codex/scripts/codex-companion.mjs --helpprints usage, confirmingmain()still runs throughisDirectExecution()Backward compatibility
buildAdversarialReviewPrompt(context, focusText)keeps its existing 2-argument signature; the only existing caller (codex-companion.mjsline ~409,const prompt = buildAdversarialReviewPrompt(context, focusText)) is unchanged.collectReviewContext()return shape is untouched; the lightweight branch reads optionalsummary/changedFiles/fileCountfields it already populates.Known follow-ups (out of scope for this PR)
These were surfaced during review and intentionally deferred:
stop-review-gate-hook.mjshas the same uncapped-prompt risk —last_assistant_messageis interpolated intostop-review-gate.mdand passed via process arguments without a byte cap. The fix shape is different (process argv vs inline string), so it warrants a separate PR.changedFilesis rendered into the lightweight summary, paths are joined with\n. Git rejects newlines in tracked paths so the practical injection surface is small, but defense-in-depth (JSON-encode or fence the list) is worth a follow-up.{ prompt, mode, originalBytes, finalBytes }and surface that in the job payload.Notes on the constant
MAX_PROMPT_BYTES = 800 * 1024(≈ 819 200 bytes) leaves a ~229 KB margin under the 1 048 576-character API cap. The API's limit is in code units whileBuffer.byteLength(..., "utf8")counts bytes, so for ASCII the margin is the full 229 KB; for the (very rare) case of an all-multi-byte prompt the margin shrinks but never to zero with realistic content.Closes cardene777/claude-config#1467